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This  booklet  Is  intended  to  assist  districts  with  the  assessment  aspects 
of  their  Basic  Skills  Improvement  Plans.  As  much  as  possible,  steps  and 
procedures  are   presented  in  a  straightforward  manner.  This  means  that 
every  possible  situation  is  not  explained  in  detail,  and  that  specific 
references  to  some  points  in  the  Regulations  for  Implementation  of  the 
Basic  Skills  Improvement  Policy  have  been  omitted.   The  Regulations,  of 
course,  are  the  "final  word"  on  requirements.     Districts  should  refer 
to  them  in  conjuction  with  this  booklet  for  a  complete  understanding 
of  the  requirements  of  the  Pol  icy. 


NTR0DUCT10N 


We've  said  it  before,  but  it  needs  to  be  said  again.   This  Policy  is 
called  the  Basic  Skills  Improvement  Policy.   It  is  intended  to  focus 
on  curriculum.   Evaluation  of  student  achievment  of  minimum  standards 
is  obviously  necessary  in  order  to  know  how  students  are  doing  and  whether 
or  not  they  have  achieved  minimum  standards.   However,  this  is  not  a  "testing 
program".   Assessment  is  one  aspect  of  implementing  this  Policy;   it  is  not 
the  whole  program.   It  is  not  even  the  biggest  part  of  the  program.   This 
booklet  focuses  only  on  the  assessment  aspects  of  the  Policy. 

In  1980-81,  students  need  to  be  evaluated  for  achievement  of  minimum 
standards  in  Reading,  Writing,  and  Mathematics.   Starting  in  1981-82, 
students  also  need  to  be  evaluted  for  achievement  of  minimum  standards  in 
Listening  and  Speaking. 

School  districts  have  different  curricula,  different  organizational 
structures,  different  overall  curriculum  objectives,  different  areas  of 
strength,  and  different  needs.   This  Policy  recognizes  this  diversity,  and 
provides  a  great  deal  of  flexibility  to  local  districts  in  developing  and 
implementing  programs  to  improve  achievement  of  basic  skills. 

At  the  ELEMENTARY  levels,  each  district  decides: 

which  grade  levels  to  pick  for  assessment. 

what  the  basic  skills  objectives  are. 

what  evaluation  instruments  are  appropriate. 

the  definition  of  achievement  of  minimum  standards  ("cut-off  scores" 

on  a  test) . 

At  the  SECONDARY  level,  districts  decide  all  of  these  things,  with  restrictions 
and  requirements  for  approval  by  the  state.   Look  at  the  chart  on  the  next 
page  for  a  summary. 
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REGULATIONS  FOR  IMPLEMENTATION  OF  THE 

BASIC  SKILLS  IMPROVEMENT  POLICY 
(January,  1979) 

Sections  on  Evaluation 


^O«03'   Establishment  of  Minimum  Standards 

(1)  Each  public  school  district  shall  establish  minimum  standards 
in  each  of  the  basic  skills  for  the  following  grade  levels,  if 
applicable:  early  elementary,  later  elementary  and  secondary. 

At  the  secondary  level  the  minimum  standards  established  by  each 
oublic  school  district  shall  include,  but  not  necessarily  be 
limited  to,  the  basic  skills  objectives  set  forth  in  Section  kQ.Qk 
of  these  regulations . 

(a)  3y  August  I,  1930,  eacn  public  school  district  shall 
establish  minimum  standards  in  the  basic  skills  of  reading, 
writing  and  mathematics. 

(b)  By  August  TT I SlTT'each  public  school  district  shall 
estaolish  minimum  standards  in  the  basic  skills  of 
listening  and  speaking. 

(2)  Each  public  school  district  snail  determine  the  soecific 
grade  or  graces  within  the  early  elementary,  later  elementary 
and  secondary  levels  for  which  minimum  standards  shall  be 

established. 

^0.05:   Evaluation  of  3asic  Skills  Achievement 

(1)  Each  public  school  district  shall  evaluate  each  student's  achieve- 
ment of  minimum  standards  at  least  once  during  tne  early  elementary, 
later  elementary  and  secondary  levels. 

(a)  Evaluation  of  student  achievement  of  minimum  standards  in 
reading,  writing  and  mathematics  shall  begin  no  later  than 
the  end  of  the  1980-31  school  year. 

(b)  Evaluation  of  student  achievement  of  minimum  standards  \n 
listening  and  speaking  shall  begin  no  later  than  the  end 
of  the  1981-32  school  year. 

(2)  Each  public  school  district  shall  determine  at  which  grade  or  grades 
at  the  early  elementary,  later  elementary,  and  secondary  levels  students 
shall  be  evaluated  for  the  achievement  of  minimum  standards,  provided 
that  secondary  level  students  shall  first  be  evaluated  no  later  than 

the  beginning  of  grade  nine. 

(3)  Once  a  public  school  district  has  determined  at  which  secondary 
grade  level  to  begin  the  evaluation  of  student  achievement  of  minimum 
standards,  it  shall  provide  for  an  evaluation  of  student  achievement 
of  minimum  standards  at  least  once  at  each  succeeding  grace  level  until 
the  student  has  achieved  the  minimum  standards.   A  secondary  level 
student  who  has  achieved  the  minimum  standards  for  a  basic  skill  neeti 
not  be  evaluated  again  in  that  skill. 


{k)     At  the  early  and  later  elementary  levels,  each  public  school 
district  shall  determine  which  evaluation  instruments  shall  be  used 
to  evaluate  student  achievement  of  minimum  standards. 

(5)  At  the  secondary  level,  each  public  school  district  shall  have  the 
option  of  using  one  or  more  of  the  following  evaluation  instruments  to 
evaluate  student  achievement  of  minimum  standards: 

(a)  Evaluation  instruments  available  from  the  Department  of  Educatior 

(b)  Commercially  available  evaluation  instruments  approved  by  the 
Department  of  Education;  or 

(c)  Locally  utilized  or  developed  evaluation  instruments  approved  by 
the  Department  of  Education  as  being  comparable  to  either  (a) 

or  (b). 

(6)  In  accordance  with  Section  4.03  of  the  Chapter  622  Regulations 
Pertaining  to  Access  to  Equal  Educational  Opportunity  as  adopted  by 
the  Board  of  Education,  a  public  school  district  shall  not  use  an 
evaluation  instrument  which  discriminates  en  the  basis  of  race,  color, 
sex,  religion  or  national  origin. 

(7)  After  an  evaluation  of  a  student's  achievement  of  minimum  standards, 
each  public  school  district  shall  give  the  student  and  his  or  her 
parent  the  opportunity  to  review  the  evaluation  instruments  used  and 

the  complete  record  of  the  student's  achievement  level.   The  complete 
record  of  a  student's  achievement  level  shall  include,  but  not  necessarily 
be  limited  to,  the  corrected  responses  of  the  student  on  tne  evaluation 
instruments  used,  and  shall  be  maintained  as  part  of  the  student's 
temporary  record  pursuant  to  the  Student  Records  Regulations  as  adopted 
by  the  Board  of  Education, 

40.10:   Transfer  Students,  Students  in  Special  Education  Programs,  and  Students 
of  Limited  English  Ability 

(1)  A  public  school  district  may  exempt  any  student  who  transfers 
into  its  school  district  after  having  comoleted  the  tenth  grade  in 
another  district  from  the  annual  reporting  requirements  of  Section  ^0.0$. 

(2)  The  evaluation  team  responsible  for  developing  a  student's 
Individual  Educational  Plan  in  accordance  with  General  Laws,  Chapter  7' 3 
and  the  Chapter  766  Regulations  shall  determine  whether  a  student  in 
need  of  special  education  shall  be  evaluated  for  achievement  of  the 
minimum  standards  provided  for  by  these  regulations,  in  addition  to  the 
objectives  contained  in  the  Individual  Educational  Plan,  and  whether 
the  result  of  such  an  evaluation  shall  be  included  in  the  public 
school  district's  annual  reoort  submitted  pursuant  to  Section  40.09. 
The  evaluation  team's  determination  shall  be  included  in  the  student's 
Individual  Educational  Plan. 

(3)  Each  public  school  district  shall  exemot,  from  all  provisions 
of  these  regulations,  students  who  have  limited  English  ability  as 
defined  by  General  Laws,  Chapter  71  A.   Notwithstanding  the  previous 
sentence,  students  who  have  completed  an  appropriate  transitional 
bilingual  education  program  in  accordance  wi th  General  Laws,  Chapter 
71A  and  the  Regulations  for  Use  in  Administering  Programs  in  Transi- 
tional Bilingual  Education  adooted  by  the  3oard  of  Education  shall 

be  considered  as  any  other  students  for  the  purpose  of  these  regulations. 


EVALUATION  OF  BASIC  SKILLS  ACHIEVEMENT  AT  THE  ELEMENTARY  LEVEL 


1.  Each  student  must  be  assessed  at  least  once  during  grades  K-3 
and  grades  **-6  in  each  basic  skill  area    (Reading,  Writing,  and 
Mathematics  beginning  in  1980-81;  add  Listening  and  Speaking  beginning 
in  198l-82)o 

2.  You  must  have  the  participation  of  parents,  teachers,  administrators, 
and  the  general  public  in  setting  minimum  standards. 

3.  You  pick  the  grade  level  for  assessment  of  each  skill  area.   (In  other 
words,  you  might  assess  reading  in  grades  2  and  4,  writing  in  grades  3 
and  6,  and  mathematics  in  grades  3  and  5«   Another  example:  you  might 
assess  reading,  writing,  and  mathematics  in  grades  3  and  5.   These  are 
just  two  possibl i t ies.   You  decide  what  makes  the  most  sense  for  your 
district. ) 

k.  Before  you  can  select  an  appropriate  instrument,  obviously  you  must 
know  what  you're  trying  to  evaluate.  So,  you  need  to  identify  your 
basic  skills  objectives  In  each  area. 

5.  You  review  existing  evaluation  instruments,  including  anything  developed 
locally.  Then,  you  select  the  instrument(s)  you  wish  to  use,  or  develop 
your  own. 

6.  You  determine  the  level  of  achievement  ("cut-off  point")  for  each 
basic  ski  1 1  area. 


i 


7.  You  report  the  number  and  percentage  of  students  who  have  and  have  not 
achieved  your  minimum  standards  for  each  basic  skill  area. 

8.  Forms  will  be  provided  for  reporting  this  information.   The  forms  will  be 
available  in  October,  1379. 


EVALUATION  OF  BASIC  SKILLS  ACHIEVEMENT  AT  THE  SECONDARY  LEVEL 


OPTION  A:  STATE  TESTS 


At  the  secondary  level,  each  district  has  three  options  for  evaluating 
student  achievement  of  minimum  standards. 

Section  40.05 

a.)     EVALUATLQH  IhlSTRlMEUTS  AVAILABLE  FROM  THE  VEPARTMENT  (?F  EDUCATION: 

b)  Commercially  available  evaluation  instruments  approved  by  the 
Department  of  Education;  or 

c)  Locally  utilized  or  developed  evaluation  instruments  approved  by 
the  Department  of  Education  as  being  comparable  to  either  (a) 

or  (b).  . 


The  Department  of  Education  has  developed  tests  to  assess  student  achieve- 
nent  of  the  secondary  level  basic  skills  objectives  in  Reading,  Writing, 
and  Mathematics. 


fhese  tests  were  administered  to  a  statewide  random  sample  of  students  in 

jlarch  and  April,  1979,  to  get  some  baseline  data  about  the  performance 

pf  secondary  level  students.   Results  will  be  available  in  September,  1979 


district  may  chose  to  use  a  state  test  for  any  or  all  of  the  basic  skill 
reas. 


attached  are  copies  of  the  state  tests.   With  each  test  is  a  listing  of 
he  state  basic  skills  objectives  and  the  items  on  the  test  which  were 
esigned  to  measure  each  objective.   This  information  is  provided  for 
everal  reasons: 


1,   To  help  clarify  the  objectives. 


2.  To  allow  districts,  if  you  wish,  to  administer  these 
tests  prior  to  the  approval  of  your  Basic  Skills  Plan 
in  order  to  conduct  a  needs  assessment. 

3.  To  allow  districts  to  see  what  the  state  tests  are  like, 
You  need  to  examine  the  test(s)  to  decide  if  you  want 

to  use  one  or  more  of  them  as  part  of  your  Basic  Skills 
Improvement  Program. 


> 


-If  Copies  of  the  state  tests  are   not  included  here,  check  with  your 
Superintendent.   Each  Superintendent  received  a  copy  of  the  tests. 


7. 


Using  these  materials,  you  can  review  the  state  tests  to  answer  such 
questions  as: 

.  Are  the  test  items  at  the  right  level  of  difficulty  for 
our  students,  or  do  we  want  to  consider  other  tests 
that  are  more  or  less  difficult? 

For  gross  screening  purposes,  are   we  satisfied  with  the 
number  of  items  for  each  objective,  or  do  we  want  to 
consider  other  tests  with  more  items  per  objective? 
(The  state  tests  are  not  intended  to  be  diagnostic  tests.). 

.  Are  the  state's  objectives  and  test  items  too  limiting 
for  us?  Should  we  consider  other  tests  that  include 
the  state  objectives  but  also  have  other  objectives 
that  correspond  to  our  local  basic  skills  objectives? 

Based  on  a  review  such  as  this,  a  district  can  determine  whether  or  not 
to  select  state  tests  as  part  of  your  Basic  Skills  Improvement  Plan. 


IF  YOUR  DISTRICT  DECIDES  TO  USE  A  STATE  TEST,  HERE  IS  WHAT  WILL  HAPPEN... 

For  each  state  test  that  you  decide  to  use  (Reading,  Writing,  and/or 
Mathematics) : 

1.  You  will  indicate  in  your  Basic  Skills  Improvement  Plan 
that  you  will  use  the  state  test. 

2.  You  will  receive  three  equivalent  forms  of  the  attached 
sample  test.  They  will  not  be  the  same  exact  tests,  but 
ones  which  have  been  constructed  to  have  parallel  content 
and  to  be  equivalent  to  the  sample  test. 

3.  You  will  receive  a  specimen  set  which  will  include: 

-Three  equivalent  forms  and  answer  keys 

-Technical  manual 

-Directions  for  test  administration 

k.      It  will  be  the  district's  responsibility  to  have  the  test 
duplicated  and  scored.   (This  can  be  done  locally,- or  by 
arrangement  with  commercial  test  publishers  who  will  be 
offering  these  services  to  districts.  You  will  need  to 
make  arrangements  with  them  about  costs,  score  information 
that  will  be  provided,  reporting  format,  etc.) 

5.   These  tests  may  be  administered  at  grades  7-12.   Since 

they  are  designed  as  basic  skills  tests  to  assess  achieve- 
ment of  the  basic  skills  objectives,  they  are   appropriate 
for  use  at  any  secondary  qrade  level . 


8. 


6.  For  the  Writing  Test,  your  district  will  have  to  provide 
assurances  that: 

All  students  doing  the'writing  sample  will  have 
the  opportunity  to  use  a  dictionary  while  writing. 

.  The  scoring  procedures  (using  holistic  or  analytical 
scoring)  assess  student  performance  of  the  state 
writing  objectives. 

The  scoring  procedures  insure  reliability  of  results 
(such  as,  by  training  readers,  having  two  independent 
ratings  on  holistically  scored  papers). 

A  specific  time  limit  is  not  required.   You  may  give  students 
the  opportunity  to  revise  their  work  before  it  is  evaluated. 

7.  If  you  use  a  state  test,  your  district  will  still  set  its 
own  level  of  achievement  (cut-off  score)  for  passing 

the  test. 


9. 


EVALUATION  OF  BASIC  SKILLS  ACHIEVEMENT  AT  THE  SECONDARY  LEVEL 


OPTION  B:  COMMERCIAL  TESTS 


At  the  secondary  level,  each  public  school  district  shall  have  the  option 
of  using  one  or  more  of  the  following  evaluation  instruments  to  evaluate 
student  achievement  of  minimum  standards. 

Section  40.05(5) 

a)   Evaluation  instruments  available  from  the  Department  of  Education; 
fa)  COMMERCIALLY  AVAILABLE  EVALUATION  INSTRUMENTS  APPROVED  BY  THE 

DEPARTMENT  0?  EDUCATION;   "o7 

c)   Locally  utilized  or  developed  evaluation  instruments  approved  by 

the  Department  of  Education  as  being  comparable  to  either  (a) 

or  (b). 


SUMMARY  REPORT  -  REVIEW  OF  COMMERCIAL  TESTS 

This  Aummafiizes  information  &nom  the,  attacked  report  o{,  the.  Public  AfacuUu 
Research  Institute.*    That  report  contain*  a  detailed  description  o£  the 
process  and  finding*  o£  the  {irst  screening  that  ms  conducted.     It  also 
contain*  copies  o£  all  fioim*  and  lists  o£  criteria  used  to  note  test*. 

Fourteen  test  batteries  were  reviewed  for  technical  characteristics, 
potential  bias,  and  content.  Tests  reviewed  initially  were  chosen  solely 
because  they  are  commonly  used  in  Massachusetts  secondary  schools.   Other 
tests  will  be  reviewed  when  submitted  as  part  of  Basic  Skills  Plans,  and, 
if  approved,  will  be  added  to  an  updated  Approved  List.   The  April  Approved 
List  is  just  a  starting  point;  it  will  be  periodically  updated.   JUST 
BECAUSE  A  TEST  IS  NOT  ON  THE  LIST  DOES  NOT  MEAN  THAT  IT  WON'T  BE  APPROVED. 
We  may  not  have  reviewed  it  yet;  or,  additional  information  may  be  required 
before  it  can  be  approved.   If  you  want  to  use  a  commercial  test  that  is 
not  on  the  Approved  List,  go  over  the  Criteria  for  Approval  of  Commercial 
Tests.   You  should  be  able  to  figure  out  if  your  test  will  meet  these 
criteria  and  be  approved. 

Requirement  1  :   Technical  Review 

A  score  of  80  percent  or  better  is  required  for  a  test  to  be  approved  on 
technical  grounds.   For  some  tests  reviewed,  missing  information  accounted 
for  scores  below  this  point.   With  additional  technical  information,  some 
of  these  tests  will  probably  be  approved  technically  and  added  to  the 
Approved  List. 


-If  the  PARI  Report  is  not  included  with  these  materials,  check  with  your 
Superintendent.   Each  Superintendent  received  a  copy  of  the  Report. 


10. 
Requirement  2;   Bias  Review 

Alt  tests  are  subject  to  an  independent  Bias  Review  conducted  by  the 
Bureau  of  Equal  Educational  Opportunity  of  the  Massachusetts  Department  of 
Education.   In  order  for  a  test  to  be  approved,  it  must  pass  the  BEEO  Bias  Review. 

Requirement  3»   Content  Review 

Reviewers  gave  a  rating  to  each  test  on  the  readability  level.   In  cases 
where  reviewers  determined  that  the  readability  level  was  too  difficult 
for  the  lowest  grade  level  covered  by  the  test,  districts  should  consFder 
whether  the  reading  level  of  the  test  is  appropriate  for  the  students 
in  their  districts  who  will  take  that  particular  test. 

Reviewers  also  matched  test  items  to  the  state  secondary  level  basic 
skills  objectives.   In  reviewing  Reading,  Vocabulary,  and  Mathematics 
tests,  we  found  that  a  limited  percentage  of  the  objectives  were  covered  by 
the  tests.   Requiring  complete  coverage  of  all  objectives  would  have 
resulted  in  virtually  eliminating  one  of  the  options  (option  B)  given  to 
districts  in  the  Regulations.   So,  a  compromise  was  reached.   Until 
September  1,  1983a  test  does  not  have  to  measure  all  of  the  state  objectives 
in  order  to  receive  approval.   But,  for  secondary  students  who  will  be  assessed 
for  the  first  time  in  1983-84,  all  evaluation  instruments  used  will  have  to 
cover  all  secondary  level  basic  skills  objectives  in  the  Regulations. 

Districts  still  need  to  review  the  test  they  wish  to  use  and  match 
items  to  the  state  objectives.   You  will  have  to  submit  information 
describing  the  review  and  what  you  find.   However,  districts  are  permitted 
to  use  commercial  tests  with  a  limited  matching  until  1983-84.   This  gives 
publishers  time  to  develop  instruments  which  assess  all  of  the  Massachusetts 
objectives,  and  gives  districts  a  viable  option  in  the  meantime. 

A  COMMERCIAL  TEST  WILL  BE  APPROVED  FOR  USE  IF: 

1.  it  receives  a  score  of  at  least  80  percent  on  the  technical 
review;  and 

2.  it  passes  a  Bias  Review  conducted  by  the  Bureau  of  Equal 
Educational  Opportunity. 

IF  YOUR  DISTRICT  WANTS  TO  USE  A  COMMERCIALLY  AVAILABLE  EVALUATION 
INSTRUMENT  THAT  IS  ON  THE  "APPROVED  LIST  "  HERE  IS  WHAT  WILL  HAPPEN... 

For  Reading  and  Mathematics: 

1.  You  will  indicate  in  your  Basic  Skills  Improvement  Plan  that  vou 
wish  to  use  a  specific  commercially  available  evaluation  instrument, 
and  that  this  test  has  already  been  approved  by  the  Department. 

2.  When  you  submit  your  Basic  Skills  Plan,  you  will  also  submit 
an  analysis  showing  the  content  match  between  items  on  the 
test  and  the  state  objectives,  and  assurances  that  the  reading 
level  is  suitable  for  students  in  the  lowest  grade  to  be 
evaluated. 
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MOTE:     kt  tifvu  time.,  a  dLUtAicX  may  u&e.  an  inAt/ument  even  i&  aJUL 
o£  the.  4 tote.  obje,cXiveA  one.  not  a44e44ed.     HoweveA,  ait  btjate. 
objectives  will  Hjolvq.  to  be  a64ei4ed  by  any  inAtswme.nt  used  [fan. 
the.  initial  a6AeA4me.nt  at  the.  Azcondany  IzveZ)  &tcvuting  in  1983-84. 
In  othen.  woidA,  i£  you  have,  a  commeACAol  test  which  you  axe.  aVizady 
using,   but  wkick  does  not  a£4e44  alt  o&  the.  Atate.  objectives,   you. 
may  continue,  to  use.  it  until.  September.  7,    1983,  as  long  as  it  meets 
the.  othen.  n,equin.ements  fan.  approval.     By  September  1 ,   1983,  koweven., 
you/i  instrument  [a]  must  assess  aJUL  oi  the  Atate  objectives. 


3.  You  will  not  have  to  submit  copies  of  the  instrument (s)  or 
technical  information,  since,  if  the  test  has  been  reviewed 
and  approved,  we  already  have  all  the  necessary  information 

k.      Forms,  available  in  October,  1979,  will  be  provided  for 
submitting  information  noted  above. 
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For  Writing: 

The  Department  of  Education  grants  approval  for  public  school  districts 
to  select  any  commercially  available  instrument  which  assesses  student 
writing  by  means  of  a  writing  sample  as  long  as  assurances  are  provided 
that: 

1.  All  students  doing  the  writing  sample  will  have  the 
opportunity  to  use  a  dictionary  while  writing. 

2.  For  the  writing  item(s)  selected: 

Each  item  is  clear  and  concise. 
The  directions  are  specific. 
The  topic  allows  for  a  range  of  responses. 
The  content  should  not  limit  the  ability  of  an 
average  student  to  write  an  acceptable  response* 
The  vocabulary  is  not  too  difficult  to  be  under- 
stood by  an  average  student. 

Each  item  indicates  the  way  the  student  should  proceed 
(give  your  reasons,  citespecific  examples,  etc.). 
Each  item  is  within  the  range  of  experience  of  all 
students.   (The  content  should  not  provide  an  unfair 
advantage  or  disadvantage  to  any  group  of  students.) 
Each  item  is  free  of  offensive  sexual,  cultural,  racial 
and/or  ethnic  content  and/or  stereotyping. 

3.  The  scoring  procedures  (using  holistic  or  analytical  scoring) 
assess  student  performance  on  the  state  writing  objectives. 

h.      The  scoring  procedures  insure  reliability  of  results  (such  as, 
by  training  readers,  having  two  independent  ratings  on 
holistically  scored  papers). 


The  stimulus  for  a  student  writing  sample  need  not  be  limited  to  a  written 

question  or  questions.  A  visual  stimulus,  such  as  a  photograph,  may 

be  used,  as  long  as  the  school  district  can  provide  the  required  assurances. 

A  specific  time  limit  is  not  required.   You  may  give  students  the  opportunity 
to  revise  their  work  before  it  is  evaluated. 


\k. 


IF  YOUR  DISTRICT  WANTS  TO  USE  A  COMMERCIALLY  AVAILABLE  EVALUATION 
INSTRUMENT  THAT  HAS  NOT  BEEN  APPROVED  BY  THE  STATE  YET,  HERE  IS  WHAT 
WILL  HAPPEN... 


For  Reading  and  Mathematics: 

1.  You  will  indicate  in  your  Basic  Skills  Improvement  Plan  that 
you  wish  to  use  a  specific  commercially  available  evaluation 
instrument  (or  instruments). 

2.  When  you  submit  your  Basic  Skills  Plan,  you  will  also  submit  a 
copy  of  the  test;  an  analysis  showing  the  content  match  between 
items  on  the  test  and  the  state  objectives;  assurances  that  the 
reading  level  is  suitable  for  students  in  the  lowest  grade  to  be 
evaluated;  and  supporting  technical  information  (see  next  page). 

MOTE:     At  thi*  time.,   a  di*t/vict  may  u*e  an  in*tnwment  even  i&  atl  o£  the  Atate  objective* 
axe  not  a**e**ed.     HoweveA,  all  ttate  objective*  will  have  to  be  a**e*Aed  by  any 
iyibt/iument  u*ed  [fio/i  the  Initial  aue*4ment  at  the.  Aecondany  level)  6ta/vtuig  in  1983-84. 
In  otken.  wond*f  i{  you  have  a  commercial  te*t  which,  you.  axe.  already  u*ing,   bat  which  doe* 
not  a*Ae*A  all  o{  the.  ttate  objective*,   you  may  continue  to  u*e  it  until  September  1 ,  1983, 
a*  long  a*  it  meet*  the  other  requirements  far  approval.     By  September  1 ,   1983,  however, 
youA  instrument^)  mu*t  a**e*4  all  o&  the  Atate  objective*. 

3.  In  order  for  your  test  to  be  approved,  it  must: 

receive  a  score  of  at  least  80  percent  on  the 
Technical  Review;  and 

pass  a  Bias  Review  conducted  by  the  Bureau  of 
Equal  Educational  Opportunity. 

k.      The  technical  criteria,  and  the  points  awarded  for  each  item, 
are  given  on  the  "Standardized  Achievement  Test  Review  Form" 
which  fol lows. 

5-   If  your  evaluation  instrument  is  approved,  you  will  be  notified 
along  with  approval  of  your  Basic  Skills  Plan. 

6.   If  your  evaluation  instrument  is  not  approved,  you  will  be  told 
why.   If  you  can  provide  additional  information,  you  can  resubmit 
your  plan  for  approval  .  (See  Section  ^0.08,  Paragraph  5  of  the  Regulations.) 

For  Writing: 

The  Department  of  Education  grants  approval  for  public  school  districts 
to  select  any  commercially  available  instrument  which  assesses  student 
writing  by  means  of  a  writing  sample  as  long  as  assurances  are   provided 
that: 

1.  All  students  doing  the  writing  sample  will  have  the 
opportunity  to  use  a  dictionary  while  writing. 

2.  For  the  writing  item(s)  selected: 

Each  item  is  clear  and  concise. 

The  directions  are  specific. 

The  topic  allows  for  a  range  of  responses.   The  content 

should  not  limit  the  ability  of  an  average  student  to 

write  an  acceptable  response. 
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The  vocabulary  is  not  too  difficult  to  be  understood 
by  an  average  student. 

Each  item  indicates  the  way  the  student  should  proceed 
(give  your  reasons,  cite  specific  examples,  etc.). 
Each  item  is  within  the  range  of  experience  of  all 
students.   (The  content  should  not  provide  an  unfair 
advantage  or  disadvantage  to  any  group  of  students.) 
Each  item  is  free  of  offensive  sexual,  cultural,  racial 
and/or  ethnic  content  and/or  stereotyping. 

3-   The  scoring  procedures  (using  holistic  or  analytical  scoring) 
assess  student  performance  on  the  state  writing  objectives. 

k.      The  scoring  procedures  insure  reliability  of  results  (such  as, 

by  training  readers,  having  two  independent  ratings  on  holistically 
scored  papers) . 


The  stimulus  for  a  student  writing  sample  need  not  be  limited  to  a  written 
question  or  questions.   A  visual  stimulus,  such  as  a  photograph,  could  may 
be  used,  as  long  as  the  school  district  can  provide  the  required 
assurances. 

A  specific  time  limit  is  not  required.   You  may  give  students  the  opportunity 
to  revise  their  work  before  it  is  evaluated. 


STANDARDIZED  ACHIEVEMENT  TEST  REVIEW  FORM 
Technical  Considerations 

Tku  n.evi.ew  fa/un  toaa  pn.epan.ed  by  Ron  Hambleton,  Geonge  ModcrnA  and  Veten. 
AiAOAian  to  meet  Apeci^-lcatixinA  nequiAed  by  the  Commonwealth  o£ 
MaA4<zdia6e£t6  fan.  uae  in  conjunction  with  the  MaAAachtL&ettA  Bcu>i.c  SkiZlb 
Impnjovement  Policy.     Tki6  fanm  AummaAA.ze6  infanmation  £nom  £>evenaJL  fanm* 
in  the  appended  nepont,   "Development  and  Application  o£  Znitenla.  fan 
Screening  CommencXal  Standa/uLLzed  TeitA  fan  the  MaAAachuAettA  Ba&ic  SkiZlb 
Improvement  Poticy".     Vou.  should  ne^en.  to  that  nepont  fan.  mone  detailed 
infanmation,  dei>cni.'ption6 ,  and  direction*. 

1.  How  many  alternate  forms  of  this   test  are  avai lable? Cno  points) No.   of  forms 

2.  Is   there  a  Technical    Manual   which   includes    information  about 
the   test   regarding   the  following   ten   topics:    (1    point  each) 

a.  Item  Review  Methods YES  NO 

b.  Item  Analysis YES  NO 

c.  Average    Item  Difficulty YES  NO 

d.  Internal    Consistency  Reliability    YES  NO 

e.  Test/Retest   Reliability    YES  NO 

f.  Parallel    Form  Reliability YES  NO 

g.  Standard   Error  of  Measurement YES  NO 
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h.   Content  Validity YES    NO 

i.   Norms YES    NO 

j.   Procedures  for  screening  items  for  offensive  sexual, 
cultural,  racial,  and/or  ethnic  content,  and/or 
stereotyping YES    NO 

3.   How  many  of  the  items  reviewed  meet  the  standard  rules 

of  item  writing?  (up  to  5  points)  (See  page  21,)  %   of  acceptable 

i  terns 

k.      Were  item  analysis  results  used  to  identify  "defective" 

test  items?  (up  to  3  points)  YES   NO    INA* 

5.  Are  data  bearing  on  the  consistency  of  mastery  decisions 
(for  one  or  more  performance  standards  or  cut-off 

scores)  reported  in  the  Technical  Manual?  (1  point)        YES   NO 

6.  Is  the  consistency  of  mastery  decisions  (for  one  or 
more  cut-off  scores)  reported  in  the  Technical  Manual 

equal  to  or  above  90%?  (5  points)**  YES   NO    INA 

7.  Do  standard  indices  of  internal  consistency  reliability 
reported  on  the  total  reading  score  or  total  mathe- 
matics score  reach  or  exceed  .90?  (up  to  5  points)         YES   NO   INA 

8.  Do  standard  indices  of  test-retest  or  parallel  form 
reliability  as  reported  on  the  total  reading  score 
or  total  mathematics  score  reach  or  exceed  .90? 

(up  to  5  points)  YES   NO    INA 

9.  If  parallel-forms  of  the  Test  are  available,  do 
both  forms  (or  multiple-forms,  if  available)  measure 
equally  well  the  content  spanned  by  the  skills 
included  in  the  Test?   (In  other  words,  do  the 
multiple-forms  of  the  Test  have  equivalent  content 

validity?)   (no  points)  YES   NO    INA 

10.  Are  the  test  score  norms  based  on  data  that  is  no 

more  than  five  years  old?   (2  points)  YES   NO   INA 

11.  Were  the  norm  groups  of  sufficient  size  (i.e.  at 

least  300  students)?   (2  points)  YES   NO    INA 

12.  Were  the  samples  of  students  used  in  the  norming 
study  representative  of  students  in  the  grades  for 
which  this  test  is  intended? 

(3  points)  YES   NO    INA 

13.  Were  the  samples  of  students  used  in  the  norming 
study  representative  of  important  strata  within  the 
society  (i.e.,  rural  pupils,  minority  group  pupils, 

pupils  in  large  city  schools,  etc.)   (3  points)  YES   NO    INA 

"INA  -  Information  not  available 

^Currently  not  included  in  tabulation  of  total  points  of  which  percentage  is 
calculated 
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14.  Are  the  test  administration  directions  suitable  for  students 
in  the  lowest  grade  covered  by  the  test? 

(2  points)  YES   NO 

15.  Do  the  test  administration  directions  address  the  matter 

of  time  limits?   (2  points)  YES   NO 

16.  Do  the  test  administration  directions  indicate  to  the 

student  how  to  handle  the  problem  of  guessing?  (2  points)       YES   NO 

17-   Is  the  layout  or  format  of  the  test  booklet  convenient 
for  students  in  the  lowest  grade  covered  by  the  test? 
(2  points)  YES   NO 

18.  Is  the  layout  or  format  of  the  answer  sheet  convenient 
for  students  in  the  lowest  grade  covered  by  the  test? 

(2  points)  YES   NO 

19.  Does  the  test  include  practice  questions?  (2  points)  YES   NO 
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EVALUATION  OF  BASIC  SKILLS  ACHIEVEMENT  AT  THE  SECONOARY  LEVEL 

OPTION  C:  LOCAL  TESTS 


At  the  secondary  level,  each  district  has  three  options  for  evaluating 
student  achievement  of  minimum  standards. 


Section  40.05 


(a)  Evaluation  instruments  available  from  the  Department  of  Education; 

(b)  Commercially  available  evaluation  instruments  approved  by  the 
Department  of  Education;  or 

.(c)  LOCALLY  UTILIZED  OR  DEVELOPED  EVALUATION  INSTRUMENTS  AVWOVED 
BY  THE  DEPARTMENT  OF  EDUCATION  AS  BEING  COMPARABLE  TO  EITHEJT 
(a)   OH   [51 


IF  YOUR  DISTRICT  WANTS  TO  USE  A  LOCALLY-DEVELOPED-  EVALUATION  INSTRUMENT, 
HERE  IS  WHAT  WILL  HAPPEN... 

1.  You  will  indicate  in  your  Basic  Skills  Improvement  Plan  that 
you  wish  to  use  a  locally  developed  evaluation  instrument 
(or  instruments) . 

2.  For  each  instrument,  you  will  submit  a  copy  of  the  evaluation 
instrument;  supporting  technical  information; and  an  analysis 
showing  the  content  match  between  the  instrument  and  the 
state  objectives. 

3.  Forms  will  be  provided  for  this.   They  will  be  available  in 
October,  1979- 

k.      The  Department  of  Education  wi 1 1  examine  the  materials  you 

submit  and  consider  your  evaluation  instrument (s)  for  approval. 

5.  The  Department  will  review  your  instrument (s)  following  a 
procedure  similar  to  the  procedure  used  for  commercial  tests. 
The  review  will  be  based  on  the  criteria  which  follow. 

6.  If  your  evaluation  instrument  is  approved,  you  will  be 
notified  along  with  approval  of  your  Basic  Skills  Plan. 

7.  If  your  evalution  instrument  is  not  approved,  you  will  be  told 
why.   If  you  can  provide  additional  information,  you  can  resubmit 
your  plan  for  approval.   (See  Section  40.08,  Paragraph  5  of  the 
Regulations. ) 
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CONTENT  CRITERIA  FOR  EVALUATING  LOCAL  ASSESSMENT  INSTRUMENTS 

1.  Evidence  that  the  district  has  examined  its  instrument (s) 
and  matched  items  to  the  state  basic  skills  objectives. 

MOTE:  A  locally  dzvzlopzd  evaluation  JjiAtiumznt 
cuA/izntly  in  o<se  ok  dzvzlopzd  pKioK  to  SzptzmbzK   J,  J  979  . 
may  be  oaed  until  SzptzmbzK  1 ,   79S3,  even  H  all  o{  thz 
6tcutz  objzctivzti  okz  not  o44e4-<sed  by  thz  inbtKumznt  [aA 
long   04  it  mzztA  thz  Tzcnnical  and  &ia6   Review  CKitZKia 
$on.  approval) . 

Any  locally  dzvzlopzd  evaluation  in&tKumznt  dzvzlopzd 
a{tZK  SzptzmbzK   J,  1979  mil  havz  to   044e44  all  o&  thz 
Atatz  objective*  bzqjjwing  uiitk  thz  ftjjut  yzaK  o{ 
aAAZ&Amznt,  and  mzzt  thz  Tzcknlcal  and  Sou   Review 
C'UtzKia  &0K  appKoval  by  tkz  VzpaKtmznt  o&  Education. 

2.  Assurances  that  the  reading  level  of  the  instrument  is  suitable 
for  students  in  the  lowest  grade  to  be  evaluated. 


3.   The  extent  to  which  the  instrument  is  free  of  offensive  sexual, 
cultural,  racial  and/or  ethnic  content  and/or  stereotyping. 
(All  instruments  are   subject  to  approval  by  the  Bureau  of 
Equal  Educational  Opportunity  of  the  Massachusetts  Department  of 
Education.) 

TECHNICAL  CRITERIA  FOR  EVALUATING  LOCAL  ASSESSMENT  INSTRUMENTS 

1.  Have  you  provided  satisfactory  information  about  these  topics? 

methods  of  developing,  testing,  and  reviewing  items 

methods  of  analyzing  test  item  results 

methods  of  determining  item  difficulty/discrimination 

methods  of  determining  test  reliability 

methods  of  determining  parallel  form  reliability 

determination  of  standard  error  of  measurement 

methods  of  screening  items  for  offensive  sexual,  racial, 

cultural,  and/or  ethnic  content  and/or  stereotyping 

2.  Were  item  analysis  results  used  to  identify  defective  test  items? 

3.  Do  test  items  meet  standard  rules  of  item  writing? 
(See  page  2T.) 

k.      Are  data  bearing  on  the  consistency  of  mastery  decisions  across 
forms  (for  one  or  more  cut-off  scores  determined  by  the  district) 
reported?   Is  the  consistency  at  least  90  %7 


5.  Are  parallel  forms  available?  Do  they  have  equivalent  content 
val idi ty? 

6.  Is  reported  reliability  at  least  90%? 

7.  Are  the  test  administration  directions  suitable  for  students  in  the 
lowest  grade  who  will  be  evaluated?   Do  they  address  the  matter 

of  time  limits?   Do  they  indicate  to  students  what  to  do  about 
guessing? 
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8.  Is  the  layout  of  the  test  booklet  and  answer  sheet  convenient 
for  students  in  the  lowest  grade  who  will  be  evaluated? 

9.  Does  the  instrument  include  practice  questions? 

CRITERIA  FOR  EVALUATING  WRIT fHG  TESTS  '  ' 

The  Department  of  Education  grants  approval  for  public  school  districts 
to  use  any  locally  developed  instrument  which  assesses  student  writi-ng 
by  means  of  a  writing  sample  as  long  as  assurances  are  provided  that: 

1.  All  students  doing  the  writing  sample  will  have  the  opportunity 
to  use  a  dictionary  while  writing. 

2.  For  the  writing  item(s)  selected: 

Each  item  is  clear  and  concise. 
The  directions  are   specific. 

The  topic  allows  for  a  range  of  responses.   The  content 
should  not  limit  the  ability  of  an  average  student  to  write 
an  acceptable  response. 

The  vocabulary  is  not  too  difficult  to  be  understood  by  an 
average  student. 

Each  item  indicates  the  way  the  student  should  proceed  (give 
your  reasons,  cite  specific  examples,  etc.). 
Each  item  is  within  the  range  of  experience  of  all  students. 
(The  content  should  not  provide  an  unfair  advantage  or  dis- 
advantage to  any  group  of  students.) 

Each  item  is  free  of  offensive  sexual,  cultural,  racial  and/or 
ethnic  content  and/or  stereotyping. 

3.  The  scoring  procedures  (using  holistic  or  analytical  scoring) 
assess  student  performance  on  the  state  writing  objectives. 

k.      The  scoring  procedures  insure  reliability  of  results  (such  as, 

by  training  readers,  having  two  independent  ratings  on  holistically 
scored  papers) . 


The  stimulus  for  a  student  writing  sample  need  not  be  limited  to  a  written 
question  or  questions.   A  visual  stimulus,  such  as  a  photograph,  may  be 
used,  as  long  as  the  school  district  can  provide  the  required  assurances. 

A  specific  time  limit  is  not  required.   You  may  give  students  the  opportunity 
to  revise  their  work  before  it  is  evaluated. 
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Multiple-Choice  Item  Writing  Principles 

1.  Is  the  item  stem  clearly  written  for  the  intended  group  of  students? 

2.  Is  the  item  stem  free  of  irrelevant  material? 

3.  Is  a  single  problem  clearly  defined  in  the  item  stem? 

4.  Are  the  answer  choices  clearly  written  for  the  intended  group  of  students? 

5.  Are  the  answer  choices  free  of  irrelevant  material? 

6.  Is  there  a^  correct  answer  or  a  clearly  best  answer? 

7.  Have  words  like  "always,"  "none,"  or  "all"  been  removed? 

8.  Are  likely  student  mistakes  used  to  prepare  incorrect  answers? 

9.  Is  "all  of  the  above"  avoided  as  an  answer  choice? 

10.  Are  the  answer  choices  arranged  in  a  logical  sequence  (if  one  exists)? 

11.  Was  the  correct  answer  randomly  positioned  among  the  available  answer  choices? 

12.  Are  all  repetitious  words  or  expressions  removed  from  the  answer  choices 
and  included  in  the  item  stem? 

13.  Are  all  of  the  answer  choices  of  approximately  the  same  length? 

14.  Do  the  item  stem  and  answer  choices  follow  standard  rules  of  punctuation 
and  grammar? 

15.  Are  all  negatives  underlined? 

16.  Are  grammatical  cues  between  the  item  stem  and  the  answer  choices, 
which  might  give  the  correct  answer  away,  removed? 

17.  Are  letters  used  in.  front  of  the  possible  answer  choices  to  identify  them? 

18.  Have  expressions  like  "which  of  the  following  is  not"  been  avoided? 


ALTERNATIVE  APPROACHES  TO  ASSESSMENT 

The  criteria  described  in  the  previous  pages  apply  specifically  to  the 
evaluation  of  standardized,  multiple  choice  tests  (with  the  exception  of 
criteria  for  evaluating  writing  by  means  of  a  writing  sample).   This  focus 
exists  because  that's  the  direction  many  districts  ar^    likely  to  take  in 
assessing  student  achievement. 

However,  some  school  districts  may  be  considering  alternative  measures 
of  student  acnievement  (e.g.  performance  criteria).   This  is  a  difficult  task 
to  accomplish  successfully.   However,  the  Policy  does  allow  for  this  type 
of  assessment.   Similar  technical  requirements  will  have  to  be  met,  and  the 
evaluation  approach  will  be  subject  to  approval  by  the  3ureau  of  Equal 
Ecucational  Opportunity. 

Districts  considering  alternative  approaches  to  assessment  are 
encouraged  to  discuss  their  plans  with  mempers  of  the  3ureau  of  Reasearch 
and  Assessment. 
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The  Commonwealth  of  Massachusetts 
Department  of  Education 


31  SL  James  Avenue,  Boston,  Massachusetts  021  IS 


March  22,  1979 
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MEMORANDUM 


TO:  Bureau  of  Research  and  Assessment 

"ROM:         Charles  Glenn,  Director 

Marlene  Godfrey,  Project  Director  for  Chaoter  62 
Bureau  of  Equal  Educational  Opportunity 

RE:  Bias  Review  of  Commercial  Tests  *or  the  3asic  Skills 

improvement  Policy 


Pursuant  to  Federal  legislation  (.Title  IX  of  the  Education  Amendments  of  1572, 
Section  36.36b  and  Section  36.36c)  and  State  legislation  (Chaoter  622  of  the 
Acts  of  1971,  Section  2.04,  Section  k.Q3   and  Section  3.01): 

The  3ureau  of  Equal  Educational  uoportunitv  undertook  an  indeoenaent  review  for 
bias  of  commercial  tests  being  considered  for  aDproval  under  the  Commonwealth's 
3asic  Skills  Improvement  Policy.   The  review  team,  which  was  comoosed  of  the 
Chapter  622  team,  reviewed  only  those  sections  of  the  test  being  consiaerea  for 
use  in  conjunction  with  the  Policy. 

The  review  was  conducted  on  the  content  of  the  test  instruments  -  that  is,  the 
test  items  themselves.   For  purposes  of  the  Basic  Skills  imcrovement  Policv, 
questions  of  the  aoproor lateness  of  the  norming  population  were  not  considered. 
This  review  and  any  test  aDoroval  arising  from  it,  therefore,  aooiy  only  to  the 
acceptability  of  the  items  and  overall  content.   The  assumotion  behind  this 
review  is  that  scores  will  be  reported  in  terms  of  the  percentage  of  items  a 
student  answers  correctly.   Since  the  apprcDr  lateness  of  the  norming  cooulation 
was  not  evaluated,  approval  aoes  not  include  use  of  normec  scores  (.such  as  grace 
equivalent  scores).   This  is  not  to  suggest  b!an«et  disapproval,  cut  merely  to 
state  that  this  issue  was  not  taken  into  consideration  for  surooses  of  this 
^articular  review. 

The  instruments  were  reviewed  for  overall  bias,  and  special  attention  was  given 
to  identification  of  any  items  which  might  be  offensive  to  anv  grouo. 

The  attached  naterials  provide,  further  information  on  the  bias  review: 

ADpencix  A:        Federal  and  State  Legislation 
Appendix  3:        Guidelines  Used  in  the  Review 


m   following  tests  are  accepted  by  the  Bureau  of  Equal  Educational  Opportunity 
1  being   Satisfactorily  ttqq   of  offensive  sexual,  cultural,  racial,  and/or 
thnic  content  and/or  stereotyping.   As  long  as  these  tests  meet  whatever  other 
riteria  may  be  established  by  the  Department  of  Education  for  approval,  3EE0 
pproves  these  tests  for  use  in  evaluating  secondary  level  students  pursuant 
o  the  Policy  on  Basic  Skills  Improvement: 

Basic  Skills  Assessment  Program  (Reading  and  Math),  1977 

California  Achievement  Tests  (Reading  and  Mathematics;  Levels  17,  18,  13),  1977 

Comprehensive  Test  of  Basic  Skills  (Reading  and  Mathematics,  Levels  3  and  4)  1 973 

Cooperative  Mathematics  Test  (Arithmetic),  1962 

Gates  MacGinitie  (Reading  and  Vocabulary;  Levels  E  and  F) ,  1978 

Iowa  Test  of  Basic  Skills  (Reading,  Vocabulary,  and  Mathematics;  Levels  13 

and  \k)  ,  1978 

Iowa  Test  of  Educational  Development  (Mathematics),  1972 
Metropolitan  Achievement  Test  (Reading  and  Mathematics;  Advanced  1  and  Advanced  2), 

1978 
SRA  Achievement  Series  (Reading  and  Mathematics;  Levels  F,  G,  H)  ,  1973 
Stanford  Achievement  Test  (Reading  and  Mathematics),  1372 
Stanford  Diagnostic  Reading  (Reading  Comprehension,  Brown  Level),  1976 
Standord  Test  of  Academic  Skills  -  TASK  (Reading  and  Mathematics;  Levels  1A 

and  11A) ,  1972 
S'EP:   Secuential  Test  of  Educational  Progress  (Reading  and  Math  Concepts, 

Level  I/Math  Confutation,  Level  IJ),  1973 

t  should  be  noted  that  none  of  these  instruments  were  designed  specifically  for  the 
Isic  Skills  Policy  of  the  Commonwealth,  but  were  drawn  up  to  meet  more  general 
ojectives.   In  many  cases,  only  one  or  two  sub-tests  from  a  test  battery  were 
Ir.sidered,  as  only  the  objectives  of  the  Basic  Skills  Policy  in  Reading  anc 
athematics  were  of  concern.   IN  NO  WAY  SHOULD  THE  COMMENTS  OF  THE  REVIEW  TEAM 
E  CONSIDERED  AS  AN  EVALUATION  OF  THE  TOTAL  INSTRUMENT  FOR  USE  OUTSIDE  THE 
KU I REMENTS  OF  THE  BASIC  SKILLS  IMPROVEMENT  POLICY. 

Istruments  may  or  may  not  have  additional  sections  which  reflect  the  same  degree 
f  bias,  or  ncn-oias,  as.  the  sections  reviewed.   The  norming  data  may  also  create 
iditional  concerns,  especially  for  systems  which  intend  to  use  the  DuDlishers1 
:rms  for  ourposes  other  tnan  for  Basic  Skills  Improvement  Programs.   Under  state 
id  "ederal  regulations,  school  systems  are  responsible  for  their  own  review  of 
ny  instrument  they  intend  to  use  to  ensure  that  it  is  not  biased  (see  Acpendix  A). 


25, 
APPENDIX  A;   Federal  and  State  Leaislation 


Title  !X  of  the  Education  Amenaments  of  1S72: 
Section  36.36b 

...testing  or  other  materials  for  aopraisfng  or  counseling  stucents  shall 
not  use  different  materials  for  students  on  the  basis. of  their  sex  or  use 
materials  which  permit  or  require  different  treatment  of  students  on  such 
oasis  unless 

...the  use  of  such  different  materials  is  shown  to  be  essential  to 
eliminate  sex  bias.   Where  the  use  of  a  counseling  test  or  other  instrument 
results  in  a  Substantially  disproportionate  numoer  of  members  of  one  sex 
in  any  particular  course  of  study  classification,  the  recipient  shall  tai<e 
such  action  as  is  necessary  to  assure  itself  that  such  disproportion  is  not 
the  result  of  discrimination  in  the  instrument  or  its  aDolication. 

Section  36.36c 

Disproportion  in  classes.  Where  a  recioient  finds  that  a  particular  class 
contains  a  substantially  a i sproport ionate  number  cf  individuals  of  one  sex, 
the  recipient  shall  take  such  action  as  is  necessary  to  assure  itself  that 
such  a i soroport ion  is  not  the  result  of  discrimination  en  the  basis  of  sex  in 
counseling  or  appraisal  materials  or  by  counselors. 

Chapter  622  of  the  Acts  of  1571: 
Section  2 . 04 

Any  standards  used  as  part  of  the  admissions  process,  including  but  not 
limited  to  testing,  the  use  of  recommendations  anc  interviewing,  to  any  puoiic 
school  shall  not  discriminate  on  the  basis  of  race,  color,  sex,  religion  or 
'.ational  origin.   Limited  English-speaking  ability  (as  cefinec  by  Chapter  71A 
of  the  General  Laws)  shall  not  be  usee  as  a  deterrent  to  or  limitation  en 
acmi  ss  ions . 

Section  4.03 

No  materials  or  tests  snail  be  employed  for  guicance  purposes  which  riser:' 
minate  and/or  limit  choices  on  tne  basis  of  race,  coior,  sex,  religion  or 
national  origin. 


iect ion 


3.01 


The  scr.col  committee  of  eac.n  scnocl  district  snail  estaolisn  policies, 
promote  regulations  ana  orccecures,  ana  implement  monitoring  arc  evaluation 
oractices  tnat  support  ana  promote  affirmative  action  anc  stimulate  necessar 
cnanges  to  insure  that  all  costacies  to  ecua!  access  to  scnoci  programs  "or 
all  persons  regardless  of  race,  sex,  color,  national  origin,  -elig:on  or 
limited  English  -  speaking  ability,  no  matter  .new  suctle  or  un  in  ten  tec, 
are  removec. 


> 


26. 

BIAS  REVIEW  CRITERIA 
FOR 
COMMERCIALLY  AVAILABLE  AND 
LOCALLY  DEVELOPED  TESTS 

All  evaluation  instruments  will  be  reviewed  for  bias  on  the  basis 
of  these  criteria.   In  reviewing  instruments,  overall  patterns  of 
bias  and  blatantly  offensive  content  are    to  be  avoided.   (All  items 
listed  below  may  not  apply  in  every  case.) 


I.  CHECKLIST  FOR  RELEVANCY  EVALUATION 

A.   Does  the  material  accurately  reflect  the  students' 

language  experiences?  YES   NO   NA: 

3.   Dees  the  material  reflect  the  students'  own 

exoeriences?  YES   NO   NA 

C.  Does  the  material  reflect  the  aifferent  levels  (formal, 

zeep ,  situational,  etc.)  of  the  students'  culture?         YES   NO  NA 

D.  Dees  the  material  affirm  tne  worth  of  the  group  members  as 
Individuals?  YES  NO  NA 

I.      Does  the  material  reflect  the  students'  culture  as  vital 
and  dynamic:   cne  in  which  an  individual  can  grew,  be 
nourished  anc  enriched  as  a  unique  human  oeina?  YES   NO   NA 


> 


:hec:<list  for  authsnticy  evaluation 

Is  tne  material  censicered  authentic  by  students  or 

wther  memoers  of  the  cultural  group  deoicted?  YES   NO  NA 

:.   Is  comoarlscn  of  cultures  seen  in  a  positive  light? 
(i.e.  coes  not  in  any  way  imoiy  the  inferiority  of 
one  culture  to  another.)  YES  NO  NA 


Dees  tne  material  reflect  authentic  cultural  values?        YES   NO   NA 
\.    CHECKLIST  FOR  LANGUAGE  EVALUATION 

A.   I5  tne  language  used  familiar  to  tne  student?  VES   NO   NA 


>.   is  the  material's  use  of  dialects  accurate?  Are  dialects 

used  without  confusing  them  with  one  another?  YES   NO   NA 

'. .       Is  :ne  usage  of  dialects  viewed  as  an  acceotaoie  way  of 

oonrnun  i  cat  icn?  (Not  used  In  a  aerogatory  or  demeaning  way.)  YES   NO   NA 

:hecklist  FOR  CULTURAL  EVALUATION 


> 


A.   -re  different  cultures  -ortravec  without  an  obvious 

tencency  to  ceoase  one  culture  as  ooccsed  to  anotner?       VES   NO   NA 

3.   ^re  demeaning  adjectives  avoided  in  describing  on  culture 

as  ocposeo  to  another?  (Primitive,  backward,  ancient,  etc.)  YES   NO   NA 

'r  '!c;  a co  i  :  cao  1  e 


27. 

C.  Are  cultural  group  members  shewn  in  a  variety 

of  types  of  behavior?  YES    MO    NA 

D.  Is  the  group's  material  culture  represented  in  ways 
that  avoid  emphasizing  stereotyped  categorizations 

of  the  group?  YES    NO    MA 

■  E.   Are  group  members  presented.tot  as  avoid 

being  perceived  as  homogeneous  and  interchangeable 

(save  for  sex  and  age  differences)?  YES    MO    MA 

F.  Are  group  members  shewn  as  possessing  a  variety  of 

"cultural"  traits  of  character  or  belief?  YES    MO    MA 

G.  Are  cultural  views,  attitudes,  and  values  shown  as 
things  taught,  challenged,  tested,  and  sometimes 
discarded,  rather  than  as' just  naturally  shared 

and  accepted  by  all  the  group  memcers?  YES    MO    MA 

H.   Are  the  cultural  group's  relations  with  other 
groups  explained  as  resoonses  to  ne^d ,    threat, 
competition,  etc.,  instead  of  dictated  by  some 
inherent  cultural  attitude  (such  as  "hostile", 
"war! ike",  etc.)  YES    MO    MA 

k.2  Content  Relevance 

A.   Ooes  the  material  reflect  the  variety  of  attitudes, 

styles,  and  roies  within  the  culture?  YES    MO    MA 

3.   Qces  the  material  recognize  both  differences  and 

similarities  between  cultures?  YES    MO    MA 

C.  Ooes  it  reflect  the  cultural  ideals,  values  and  coals 

as  well  as  the  material  and  institutional  constructs?  YES    MC    MA 

D.  Is  the  group's  intellectual  and  artistic  expression 
represented  by  its  members  achievements  in  ^orms 
snared  by  other  groups  as  well  as  those  unicue  to 

itself?  YES     MO     NA 

E.  Are  the  group's  cultural  tra i ts (whether  material, 
artistic  or  attitudinal)  explained  as  having 
evolved  through  interaction  with  the  environment 

or  with  other  cultural  groups?  VES    NO    NA 

F.  Dees  the  material  emphasize  the  worth  of  cultur-      YES    NO    NA 
al  difference  as  well  as  similarity? 


j  . 


!s  the  material  accurate  in  Its  descriptions  and      VSS    MO    NA 
explanations  of  cultural  traits,  vaiues,  senavicr? 
(Mexicans  do  not  celebrate  tneir  Incecencence  Day 
on  the  Cinco  ce  Mayo.) 

If  the  fact  of  a  particular  culturally  ana  Unguis-   VES    NG    NA 
tically  cistinct  group's  ex i stance  within  United 
States  society  adecuately  presented?   (Native  Amer- 
icans in  New  v  r<  Citv  and  Puerto  R'cans  in  Mew 
Mexico,  for  axamola,  are  rarely  noticed.; 


I 


5.   CHECKLIST  FCR  POLITICAL  3  IAS 

A.   Does  the  material  present  an  accurate  and  objective 
account  of  tne  conflicts  between  the  United  States 
and  other  nations?  YES    NO    NA 

3.   Does  the  material  present  an  accurate  and  objective 
account  of  the  aims,  needs  and  conditions  of  political 
protest  within  the  United  States?  YES    NO    NA 

C.   Does  the  material  present  an  accurate  account  of  the 

history  of  the  democratic  form  of  government?        YES    NO    NA 

o.   CHECKLIST  FOR  RACIAL  3  I  AS 

o.l     3  las 


» 


A.   Dees  the  materia!  avoid  debasing  one  race  as  CDOosed 

to  anotner?  YES    NC    NA 

3.   Are  consistent  portrayals  of  subserviant  relationships 

between  racial  groups  avoiaed?  YES    NO    NA 

C.   Are  cersans  of  all  races  shown  as  members  of  a  variety 
of  social,  religious  or  political  groups  and  economic 
classes?  YES    NO    NA 

0.   Are  members  of  one  race  not  depicted  as  experiencing 
a  narrower  range  of  feelings  and  emotions  than  members 
\  of   another  race?  YES    NO    NA 

E.  Are   all  members  of  a  racial  group. not  oresented  as 

sharing  in  "typical"  attitudes  or  feelings?         >  ES    NO    NA 

r.  Arz   color  labels  which  are   used  to  indicate  value 

("black"  for  evil;  "-white"  for  pure,  etc.)  avoided?  YES    NO    NA 

.G.   Are  members  of  a  racial  grous  not  stereotyoed  as  to 

manner  of  speech,  style  of  dress,  pr^farred      diversions, 

etc.?  YES     NO    NA 

H.   Are  inter-racial  contacts  not  pictured  consistently 

as  dangerous  or  hostile?  YES    NO    NA 

!.  Are    persons  of  one  race  net  presented  as  having  inherent 
abilities  which  are  aeniec  to  persons  of  otner  racial 
groucs  (non-whites  better  suited  to  arutal  labor  than 
whites;   nan-whites  more  "in  tune  with  nature", 
etc.;?  YES     NO    NA 

z.2  Content  Relevance 


Dees  tne  rraterial  reflect  the  racial  diversity  of 

United  States  society?  YES    NO    NA 

! -3  an  adequate  and  objective  account  of  racial 

discrimination  cresentac?  YES    NO    NA 

Is  an  adecuate  account  of    cultural,  historical  and 

social  contributions  of  a  1 1  racial  grouDS  in 

United  States  society  presented?  VES    NO    NA 


D.   Does  Che  material  reinforce  the  value  of  interethnic 

contact?  YES     NO    NA 

£.   Are  members  of  all  racial  groups  presented  as  equally 

complex  and  worthwhile  human  beings?  YES    NO    NA 

F.   Is  the  diversity  of  individual  experience  within  a 

racial  group  adequately  represented?  YES    NO    NA 

7.   CHECKLIST  FOR  HISTORICAL  3  I  AS 

7.1     3ias 

A.   Does  the  material  avoid  suppressing  or  distorting 
the  presence,  contribution  and  point  of  view  of 

culturally  distinct  groups  to  the  United  States      YES    NO    NA 
hi  story? 

3.   Are  known  historical  factors  not  sucpressed  in  order 
to  make  the  actions  of  seme  group  or  person  seem 
good  or  bad?  YES    NO    NA 

C.  Are  different  peoples  and  cultural  grouos  presented 

as  people  struggling  to  satisfy  their  own  needs  and  defend 

their  own  values  rather  than  as  obstacles  in  the  path 

of  another  civilisation?  YES    NO    NA 

D.  Is  the  North  American  continent  not  depicted  as 
essentially  unsettled  until  the  advent  of  white 

Eurooeans?  YES    NO    NA 

E.  Does  the  materal  nor  ignore  the  cost  to  certain  groups 
of  United  States  historical  deve iopment (s i a.ery  of 
Black  people;  loss  of  land  to  Native  Americans; 

and  economic  subjugation  of  Hispanos  in  the  Southwest; 

unfair  taxation,  exploitation  and  discrimination  against 

Asian  labor,  etc.)?  YES    NO    MA 

F.  Dees  the  material  avoid  representing  the  rights,  goals 
and  needs  of  one  group  of  people  as  less  worthy  than 

those  of  another  group  in  the  United  State  history?   YES    NO    NA 

3.   CHECKLIST  FOR  RELIGIOUS  3  I  AS 

3. 1     3ias 

A.   Does  the  material  avoid  debasing  cne  religion 

as  opoosed  to  another?  YES    NO    'JA 

3.   Dees  the  material  avoid  proselytizing  r'or  a 

rel igicn  or  sect?  YES    NO    MA 

C.  Does  the  material  avoid  slighting  references  to 

"primitive"  religious  beliefs  or  ''suoerst  i :  ions:l?    YES    NO    NA 

D.  Are  religious  symools,  icons  or  rituals  net  afscussed 

in  a  derogatory  or  demeaning  manner?  VE3    NO    MA 


7* 

2* 


Content  Relevance 


A.   Does  the  material  reflect  and  encourage  equal  respect  for 

all  religious  views?  YES    MO    NA 

3.   Does  the  material  explore  the  common  elements  found 

in  various  religions?  YES    NO    NA 

CHECKLIST  "OR  SEXUAL  3  IAS 

3?as 

A.   Dees  the  material  avoid  debasing  one  sex  as 

opposed  to  the  other?  YES    NO    NA 

3.   Dees  the  material  avoid  showings  females  only 

as  background  figures  in  a  male-centered  story?      YES    NO    NA 

C.   If  females  are  central  figures  are    they  shewn  in 

roles  other  -nan  the  stereotype  of  teacher,  mother, 

nurse,  ana/or  secretary?  YES    NO    NA 

0.   Are  the  actions  of  women  as  sophisticated  and  as 

interesting  as  those  of  men?   (Are  women  shewn  using 

initiative  anc  imagination  rather  than  being  docile 

and  passive?  YES     NO    NA 

£.   Are  the  values  of  beauty,  sweetness,  domesticity, 

motnerheca  and  marriage  avoided  as  depictions  of  the 

predominant  virtues  for  females  in  the  stories?      YES    NO    NA 

".   Are  male  figures  shown  as  exoeriencing  themselves  as 
worthwhile  only  because  they  shoulder  a  female's 
buraen  or  solve  her  problems  avoided?  YES    NO    NA 

Content  Relevance 


A.   Dees  tne  material  reflect  the  conditions  and  contributions 

of  women  in  United  States  society?  YES    NO    NA 

3.   Does  tne  material  encourage  both  boys  and  girls,  and 
men  ana  women  to  see  tnemselves  as  human  ceings  witn 
3n  equal  right  to  all  benefits  and  choices  orovided  by 
society?  YES    NO    NA 

C.  Does  the  material  oresent  an  objective  account  of 

women ' s  struggle  -  both  historically  and  contemcorar i 1 y  - 

to  achieve  equality  with  men?  VE3    NO    NA 

D.  2oes  the  material  encourage  girls  to  see  themselves 

as  capaoie  of  filling  any  role  in  society?  YES    NO    N 


A 


Dees  the  materia!  present  an  accurate  picture  of  women 

in  ether  cultures,  including  tneir  contributions  to  the 

culture  ana  tneir  struggles  for  eauality  within  it?  YES    NO    NA 


10.  CHECKLIST  CF  SOCIAL  BIAS  31. 


A.   Does  the  material  avoid  debasing  cne  social  croup 

as  ooposed  to  another?  YES 

3.   Does  the  material  avoid  showing  only  the  nuclear 
family  with  middle  class  values  as  a  worthwhile 
moael  for  family  structure?  ycc 


NO     NA 


NO     NA 


C.   Do  the  material's  Illustrations   avoid  depictina 
food  and  dress  that  ara   common  only  to  middle 
class  families?  vgs    nq    \ia 

0.   Does  the  material  avoid  deoicting  persons  of  csrtain 
cultural  groups  as  being  poor  without  any  explanation 
for  the  fact?  Y£s    M  NA 

The  following  sources  were  used  in  compiling  the  above  criteria: 

Materials  from: 

The  Lau  Assistants  Center,  California 

The  State  Department  of  Education,  California 

The  Far  West  Laboratory,  California 

Issues  of  Sex  Bias  and  Sex  Fairness  in  Career  Interest  Measurements, 
H.E.W.,  National  Institute  of  Education,  1975. 


I 


